
What is claimed is: 



^ 1 . A s *qbot system having an image processing function for 
determining postu^ or posture and position of a robot operation on an object 
comprising: 

a robot; 

a first image captuAig device; 
a memory storing reference models created based on image data of a 
reference object captured by said image capturing device in a plurality of 
different directions, and storing information of the capturing directions to be 
respectively associated with said Reference models, and information of 
orientation of the robot operation tyith respect to the object, said reference 
object being the object of detectior or an object having a shape identical to 
that of the object of detection; and 

a processor to perform matching processing on image data containing 
an image of the object of detection captured by said image capturing device 



with said reference models to selec 



an image of an object matched with one of 



said reference models, and to deter nine orientation, or orientation and position 



of an operation to be performed by 
the object, said one reference mode 
direction and the information of the 



ihe robot based on the selected image of 
and the information of the capturing 
orientation of the robot operation with 



respect to the object associated wit 1 said one reference model. 

2. A robot system having an image processing function according to, 
claim 1, wherein said reference mopels are obtained form a part of the image 
data of the reference object. 
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3 . A robot sysfc^^aving an image processing function according to 
claim 1, wherein said refere^e models are obtained by processing the image 
data of the reference object. 



4. A robot system havingW image processing function according to 
claim 1, wherein said first image capturing device comprises a camera for 
capturing two-dimensional image data. 

5. A robot system having an Image processing function according to 
claim 4, wherein said image data of tme reference object are captured by said 
camera from a predetermined distance 

^^V* W). A robot systemuiaving an image processing function according to 
claim If, wherein said robot system further- comprises a second image capturing 
device and 

said robot situates saidteecond image data capturing device to have 
said determined orientation or t© have said determined orientation and said 
determined position with respecttto the object, and 

said processor processes second image data captured by said second 
image capturing device to detect position and/or posture of the object with 
respect to said second image data cJbturing device 

7. A robot system having an image processing function according to 
claim 1 , wherein said robot system further comprises a second image capturing 
device for obtaining three-dimensional position; and 

said robot situates said secondWage data capturing device to have 
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said determined orientation or to have said determined orientation and said 
determined position with respect to the object, so that said second image data 
capturing device is directed to al characterizing portion of the object; 

said processor detects three-dimensional position and/or posture of the 
object based on three-dimension^ position of said characterizing portion 
obtained by said second image capturing device. 

*1 8. A robot systemJiaving an image processing function according to 
A - claim 6 or 7 , wherein said ftfist image data capturing device is used as said 
second image data capturing device. 



9. A robot system having^ an image processing function according to 
/V claim 6 Qf-^, wherein said secondumage capturing device comprises a three- 

A 1 

dimensional visual sensor of spot-light scanning-type capable of measuring 



distance between the sensor and an 



object. 



10. A robot system having pn image processing function according to 
claim 6 of ^T; wherein said second i&age data capturing device comprises a 



structured-light unit for irradiating a structured light on an object and 
capturing an image of the object ii^luding the irradiated light on the object. 

^T^v^ 11. A robot syste^having an image processing function according to 
claimr7, wherein said robot operation is an operation of picking up at least one 
object form a plurality of objects overlapped with each other. 



12. A robot system having an image processing function for 
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determining postur^ or posture and position of a robot operation on an object 
comprising: 

a robot; 

a first image capWing device; 

a memory storingvreference models created based on image data of 
different kinds of reference objects captured by said first image capturing 
device, and storing information of the kinds respectively associated with said 
reference models, and information of orientation of the robot operation with 
respect to the object of eachlkind, each of said reference objects being the 
object of operation of each kjind or an object having a shape identical to that of 
the object of operation of each kind; and 

a processor to perform matching processing on image data containing 
an image of the object of operation captured by said first image capturing 
device with said reference models to select an-image of an object matched with 



one of said reference models, 
position of the robot operation 



and to determine orientation, or orientation and 
based on the selected image of the object, said 
one reference model, the information of the kind associated with said one 
reference model and the infon nation of the orientation of the robot operation 
respect to the object associate^ with said one reference model. 



13 . A robot system having an image processing function according to 
claim 12, wherein said referencq models are obtained form a part of the image 
data of the reference object. 



14. A robot system having an image processing function according to 
claim 12, wherein said reference models are obtained by processing the image 



data of the reference object. 



15. A robot systenmiaving an image processing function according to 
claim 12, wherein said first image capturing device comprises a camera for 
capturing two-dimensional image data. 

16. A robot system having an image processing function according to 
claim 15, wherein said image aata of the reference object are captured by said 
camera from a predetermined distance. 



claim 1*2, 



17. A robot systan having an image processing function according to 
wherein said roboj system further comprises a second image 

capturing device, 

said robot situates said ^second image data capturing device to have 
said determined orientation or to %ave said determined orientation and said 
determined position with respect t& the object, and 

said processor processes second image data captured by said second 
image capturing device to detect poskion and/or posture of the object with 
respect to said second image djata capWing device 

18. A robot system having an image processing function according to 
claim 12, wherein said robot system further comprises a second image 
capturing device for obtaining three-dimensional position; and 

said robot situates said second/image data capturing device to have 
said determined orientation or to have said determined orientation and said 
determined position with respect tor the object, so that said second image data 



capturing device is directed ta a characterizing portion of the object; 

said processor detects Wee-dimensional position and/or posture of the 
object based on three-dimensional position of said characterizing portion 
obtained by said second image capturing device. 

A robot syst^a having an image processing function according to 
- claim 17 or 1 -8; wherein said first image data capturing device is used as said 
second image data capturing dAdce. 



20. A robot system having an image processing function according to 
claim 17 or 18 ; wherein said second image capturing device comprises a three- 
dimensional visual sensor of spotlight scanning type capable of measuring 
distance between the sensor and ran object. 



21 . A robot system having an image processing function according to 
rj -claim 17 or 18, wherein said seeond image data capturing device comprises a 
structured-hght unit for irrachatmg a structured light on an object and 
capturing an image of the objecBincluding the irradiated light on the object. 

^LyJ^O^ V^22. A robot system having an image processing function according to 
claimyi8, wherein said robot operation is an operation of picking up at least 
one object form a plurality of objects overlapped with each other. 
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